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Su»ary 

The  following  questions  are  discussed  for  non¬ 
linear  systems  with  input  Gaussian  processes.  How  can 
the  systea  be  represented?  How  can  the  systea  be  i- 
dentified  froa  the  input  and  output  processes?  Does 
knowledge  of  the  way  the  systea  responds  to  a  certain 
Gaussian  input  determine  the  way  it  will  respond  to 
another  Gaussian  input  or  to  a  deterministic  input? 
Does  knowledge  of  the  systea  and  the  statistics  of  the 
output  determine  the  statistics  of  the  input? 

Introduction 


In  studying  nonlinear  systems  with  random  inputs 
one  is  faced  with  three  broad  classes  of  problems .  In 
the  first  class  the  nonlinear  systea  and  the  input  are 
known,  and  the  problea  is  to  represent  the  output 
(systea  representation)  and  to  analyze  its  statistical 
properties  (systea  analysis) .  In  the  second  class  the 
input  and  output  are  known,  or  their  joint  statistics, 
and  the  problea  is  to  identify  the  nonlinear  systea 
(systea  identification).  And  in  the  third  class  the 
nonlinear  systea  and  the  output  or  its  statistics  are 
known  and  the  problea  is  to  identify  the  input  or  its 
statistics  (inverse  problea). 

Here  we  concentrate  on  the  special  but  signifi¬ 
cant  case  where  the  random  input  is  Gaussian,  and  we 
stamarize  certain  recent  results  on  these  classes  of 
problems,  which  lead  to  several  open  questions.  The 
list  of  references  is  by  no  aeans  exhaustive  and  no 
mention  is  aade  of  results  for  non-Gaussian  inputs. 

The  nonlinear  systea  is  denoted  by  8  and  the  in¬ 
put  and  output  processes  by  X  -  (Xt,  t<T)  and 

Y  »  {Yt,  teT)  respectively  (where  the  paraaeter  set  T 

is,  say,  an  interval  on  the  real  line  and  of  course 
the  input  and  output  could  have  distinct  paraaeter 
sets).  Unless  otherwise  stated  it  will  be  assumed 
throughout  that 

(1)  the  Gaussian  input  X  is  Bean  square  continu¬ 
ous,  with  mean  zero  (for  simplicity)  and 
continuous  covariance  function  R(t,s),  and 

(2)  the  nonlinear  systea  8  is  such  that  the  out- 

2 

put  Y  has  finite  second  aoaents:  EYt  <  *», 
t  t  T. 

All  integrals  are  over  T,  which  is  thus  deleted. 

Systea  Representation  and  Analysis 

It  is  shown  in  [7]  that  for  each  t  c  T, 


Yt  '  Wt  *  j,  /•••/*»<*;*! . V  VXt„dtl'"dtn' 
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where  the  series  converges  in  quadratic  Bean,  the  in¬ 
tegrals  are  multiple  Wiener  integrals  and  each  kernel 

fn(t;0  is  a  symmetric  function  on  t"  belonging  to  a 
Hilbert  space  X2(anR)  (for  detailed  definitions  the 
reader  is  referred  to  [7]).  The  terns 


n,t 


/.../  fn(t;t1,...,tn)X  ...Xt  dtj...dtn 

l  n 


are  homogeneous  forms  in  X  of  degree  n,  and  they  are 
uncorrelated  for  distinct  values  of  n.  This  represen¬ 
tation  generalizes  the  corresponding  result  of 
Wiener's  for  white  noise  X. 

Thus  the  action  of  the  systea  8  on  the  Gaussian 
input  X  is  represented  by  the  sequence  of  (determinis¬ 
tic)  kernels  (fn),  which  depends  not  only  on  the  sys¬ 
tea  8  but  also  on  the  input  X  through  its  covariance 
function  R.  This  representation  looks  very  auch  like 
a  Volterra  kernel  expansion  with  the  important  differ¬ 
ence  that  the  multiple  integrals  are  Wiener  rather 
than  Lebesgue  integrals.  Also  the  kernels 
f  (t;t|,...,t  )  do  not  necessarily  vanish,  as  Volterra 

kernels  do,  when  soae  t^  is  larger  than  t;  but  they 

have  this  property  when  the  systea  is  nonanticipator y, 
i.e.,  when  each  Y(  depends  only  on  the  past 

(Xs,  sst,  scT)  of  X. 


Having  represented  the  output  of  the  nonlinear 
systea,  we  would  like  to  study  its  statistical  proper¬ 
ties.  The  aost  interesting  question  is  to  find  or  de¬ 
scribe  the  distribution  of  Y  or  of  Y  ,  even  for  fixed 

n 

t,  i.e.,  its  univariate  distribution.  Yj  is  of  course 

a  Gaussian  process.  The  distribution  of  the  quadratic 
fora  Yj  t  is  fully  described  through  its  characteris¬ 
tic  function  [6,11],  For  n  2  3  the  distribution  of 

Y_  .  is  not  known  but  Arveson  [1]  has  characterized 
n,t  1 

all  homogeneous  forms  of  the  saae  degree  in  white 
noise  X  which  have  the  saae  distribution,  and  it 
would  be  of  interest  to  do  this  for  other  Gaussian  in¬ 
puts  and  for  systems  with  finite  degree.  Another  re¬ 
markable  result  is  that  the  tail  of  the  (univariate) 
distribution  of  the  finite  degree  form 


Y?.EYt+ 


N 

J. 


n,t 


determines  the  degree  N  [11]. 


When  the  nonlinear  system  is  time-invariant,  in 

the  sense  that  f„(t;tr •  •  •  *V  “  fn(tl_t . V1** 

and  T  is  the  real  line,  then 


Yn,t  ‘  W1* 


,t  -t)X 
n  t. 


..X,  dt. 


.dt 


and  if  X  is  stationary,  Y  is  strictly  stationary. 
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Wiener  asked  the  following  question:  What  is  the 
class  of  all  stationary  processes  that  are  tiae- 
invariant  transformations  of  white  noise  X?  It  turns 
out  that  all  such  processes  are  strongly  nixing  [11], 
and  that  every  strictly  stationary  ergodic  process  can 
be  approximated  in  law  by  such  processes  (this  is  a 
profound  result  of  Wiener's  which  was  clarified  by 
Nisio--see  [11]].  Analogs  of  these  results  for  more 
general  stationary  Gaussian  inputs  X  than  white  noise 
are  shown  in  [7]. 

System  Identification 

The  sequence  of  kernels  {f^}  which  represents  the 

action  of  the  system  6  on  the  Gaussian  input  X  can  be 
determined  from  the  joint  statistics  (e.g.  moments)  of 
the  input  and  output  processes  X  and  Y  [8].  However, 
as  was  pointed  out,  these  kernels  describe  the  action 
of  the  system  only  on  the  Gaussian  input  X  (the  ker¬ 
nels  depend  on  the  covariance  function  R  of  X),  and 
the  following  question  arises:  Does  knowledge  of  the 
way  the  system  responds  to  one  Gaussian  input  deter¬ 
mine  its  response  to  another  Gaussian  input  or  to  a 
deterministic  input? 

It  is  reasonable  to  expect  that  knowing  how  the 
system  responds  to  a  Gaussian  input  X,  i.e.,  knowing 
the  kernels  {fn),  would  at  most  reveal  how  the  system 

responds  to  another  Gaussian  input  X'  which  is  equiva¬ 
lent  to  X,  and  to  a  deterministic  input  which  is  a 
sample  function  of  X  (or  X');  and  would  provide  no  in¬ 
formation  on  how  the  system  may  respond  to  a  Gaussian 
input  X'  which  is  singular  to  X.  This  question  is 

considered  in  [8].  It  is  shown  that  the  kernels  (f  } 

n 

in  the  representation  of  the  system  0  acting  on  the 

Gaussian  input  X  determine  the  kernels  (f'}  in  the 

n 

representation  of  the  system  acting  on  an  equivalent 
Gaussian  input  X',  provided  either  one  of  the  follow¬ 
ing  conditions  is  satisfied:  (i)  the  nonlinear  system 
is  of  finite  order,  (ii)  the  Radon-Nikodym  derivative 
of  the  two  Gaussian  processes  is  a  bounded  random  var¬ 
iable.  However  the  conjecture  is  that  this  result 
should  be  true  for  general  equivalent  Gaussian  inputs 
and  for  general  (infinite  degree)  nonlinear  systems. 

It  is  also  shown  that  for  some  sequence  Nm  -*•  «>, 
with  probability  one 


lim  /  Nm  N*  N 

r(*>  *  JZ  V  ♦  Lj.'.tkf  (t;ti . tn) 

*(t1)...x(tn)dt1...dtn} 

obtained  by  Frdchet  [3]  for  all  x(t)  in  C[a.b]  or  in 
LjFa.b]  under  the  assumption  that  the  system  is  con¬ 
tinuous,  in  the  sense  that  for  each  fixed  t,  y(t)  is  a 
continuous  functional  on  C[a,b]  or  L2[a,b], 

A  classical  property  of  "smooth"  linear  systems, 
namely  time-invariant  linear  systems  with  transfer 
function,  is  that  they  can  be  identified  from  their 
response  to  exponential  inputs:  knowledge  of  the  way 
such  a  linear  system  responds  to  exponential  inputs  of 
arbitrary  frequency  determines  its  transfer  function 
and  hence  how  the  system  will  respond  to  any  input 
with  finite  energy.  One  should  likewise  determine 
classes  of  "smooth"  nonlinear  systems  which  can  be  i- 
dentified  from  their  response  to  appropriate  Gaussian 
inputs.  The  conjecture  here  is  that  if  a  nonlinear 
system  0  when  acting  on  a  deterministic  input  in,  say, 
L2[a,b]  has  a  Volterra  input-output  relationship 

X(t)  -  kQ(t)  ♦  l  L  /.../  ytitj.-.-.t  ) 

11*1 

*(t1)...x(tn)dt1...dtn  , 

then  knowledge  of  the  way  the  system  responds  to  a 
Gaussian  input  with  strictly  positive  definite  covari¬ 
ance  function  determines  its  Volterra  kernels  {k  )  and 

n 

hence  how  the  system  will  respond  to  any  (admissible) 
deterministic  or  random  input.  This  is  shown  in  [8] 
under  some  technical  regularity  conditions  on  the  sys¬ 
tem  0,  which  are  satisfied  when  0  has  finite  degree; 
its  proof  under  no  additional  assumptions  eludes  us  at 
present. 

Inverse  Problems 

Unlike  system  analysis  and  identification  prob¬ 
lems,  where  the  nonlinear  systems  studied  are  allowed 
to  have  memory  and/or  anticipation,  the  study  of  in¬ 
verse  problems  has  so  far  been  confined  to  memoryless 
time-invariant  nonlinear  systems  0: 


lim 


I  LJ...J  h, 

n*l 


n*(t;tl'  •  •  •*  v*t,  •  •  Xt  dtr  •  dtn 

i  n 


where  L  indicates  Lebesgue  integral  and  the  kernels 

|l| 

(hn)  are  continuous  in  (tj,...,t  )  and  can  be  found 
from  the  kernels  {f„}.  Thus  knowledge  of  the  way  the 

system  acts  on  a  Gaussian  input  X  determines  how  the 
system  will  act  on  almost  all  the  sample  functions  of 
X.  This  deterministic  input-output  representation  de¬ 
pends  strongly  on  the  input  covariance  R,  since  R  de¬ 
termines  the  kernels  (hJJ)  and  (up  to  a  zero  probabili¬ 
ty  set)  the  deterministic  functions  for  which  the  rep¬ 
resentation  is  valid.  It  is  remarkable  though  that, 
under  the  extremely  weak  assumption  on  the  system  that 
its  output  has  finite  second  moment,  one  can  obtain 
for  a  small  class  of  deterministic  inputs  a  represen¬ 
tation  Identical  in  form  to  the  representation 


Yt  ■  f(Xt)  .  t  £  T  , 

where  f(x)  is  a  function  defined  on  the  real  line.  In 
this  case  the  multiple  Wiener  integrals  in  the  repre¬ 
sentation  of  Yt  become  Heimite  polynomials  in  Xt> 

GrUnbaum  [«,5]  showed  that,  when  the  Gaussian  in¬ 
put  X  is  stationary  and  has  zero  mean,  its  covariance 
function  R(t)  can  be  identified  from  the  joint  moments 
of  all  orders  of  the  output  process  Y,  for  certain 
classes  of  even  nonlinearities  f(x),  including  inter- 
val -windows  f(x)  -  l(.,a)(x).  In  [2]  it  is  shown 

that  when  the  mean  of  the  input  is  zero,  the  covari¬ 
ance  function  R(t)  can  be  constructively  identified 
for  several  classes  of  nonlinearities  from  only  the 
mean  and  correlation  functions  of  the  output  Y.  In¬ 
cluded  here  are  hard-  and  soft-limiters,  quantizers, 
even  and  odd  nonlinearities,  as  well  as  quite  general 
nonlinearities.  Also  all  Interval -windows 
f(*)  ■  '((,b)W  are  found  for  which  arbitrary  covari- 
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•ness  can  be  identified. 

The  constructive  identification  of  the  nean  func¬ 
tion  s(t)  of  the  Gaussian  input  X  froa  the  output  Man 
and  correlation  functions  is  considered  in  [9]  when 
the  input  covariance  function  R(t)  is  known.  This  is 
an  equivalent  formulation  of  the  following  interesting 
problem:  a  (nonrandom  and  unknown)  signal  s(t)  in  ad¬ 
ditive  Gaussian  noise  N  with  mean  zero  and  known  co- 
variance  function  R(x),  X(  -  (t)  *  Nt,  is  passed 

through  a  aemoryless  nonlinearity  f(x),  and  the  prob¬ 
lem  is  to  identify  the  signal  s(t)  from  the  mean  and 
correlation  functions  of  the  output  process  Y.  Note 
that  in  the  absence  of  noise  the  signal  s(t)  cannot  in 
general  be  identified  from  f[s(t)j.  It  is  shown  in 

[9]  that  arbitrary  signals 

(i)  can  be  identified  when  the  nonlinearity  is 
nonotonic,  such  as  a  hard-  or  soft-limiter  or 
an  infinite-interval-window; 

(ii)  can  be  identified  up  to  a  global  sign  when 
the  nonlinearity  is  symaetric  around  som 
point  Xq,  bounded  below  or  above,  and  aono- 

tonic  on  [*0,“),  such  as  a  full  wave  even 
vth-law  device  or  a  finite- interval-window. 

The  problem  of  identifying  both  the  Man  function 
s(t)  and  the  covariance  function  R(t)  of  the  input  X 
remains  open  at  present. 

(i)  implies  in  particular  that  a  signal  can  be  1- 
dentified  from  the  first  two  moment  functions  of  the 
hardlimited  version  of  the  signal  plus  noise: 

Yt  »  sgn[s(t)  ♦  Nt].  The  question  thus  arises  whether 

the  signal  s(t)  can  actually  be  estiMted  from  the  bi¬ 
nary  output  Yt>  Note  that  in  the  absence  of  additive 

noise  the  signal  s(t)  cannot  be  identified  or  estiMt¬ 
ed  from  the  binary  output  sgn[s(t)] .  It  is  shown  in 

[10]  that  by  deliberately  adding  noise  (N. }  to  the 

k  * 

periodic  samples  of  the  signal  (s(g)}  prior  to  hard- 
limiting,  the  signal  s(t)  can  be  estiMted  consistent¬ 
ly  from  the  binary  sequence  (sgn[s(£)  ♦  NkJ)  as  the 
sampling  rate  N  tends  to  infinity.  The  estiMte  *g(t) 

is  in  fact  shown  to  converge  to  the  signal  s(t)  with 
probability  one  and  also  to  be  asymptotically  normal . 
The  estiMtor  consists  of  a  time-varying  linear  system 
followed  by  a  memoryless  tiM-invariant  nonlinearity, 
which  can  be  made  linear  by  a  proper  choice  of  the 
noise  distribution!  These  results  hold  for  all  bound¬ 
ed  and  uniformly  continuous  signals  and,  in  addition 
to  the  hard limiter,  for  further  monotonic  and  nonmono¬ 
tonic  nonlinearities. 
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